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Abstract 

The Hall effect, the anomalous Hall effect and the spin Hall effect are fundamental transport 
processes in solids arising from the Lorentz force and the spin-orbit coupling respectively. The 
quantum versions of the Hall effect and the spin Hall effect have been discovered in recent years. 
However, the quantized anomalous Hall (QAH) effect has not yet been realized experimentally. In 
a QAH insulator, spontaneous magnetic moments and spin-orbit coupling combine to give rise to 
a topologically non-trivial electronic structure, leading to the quantized Hall effect without any 
external magnetic field. In this work, based on state-of-art first principles calculations, we predict 
that the tetradymite semiconductors Bi2Te3, Bi2Se3, and Sb2Te3 form magnetically ordered insu- 
lators when doped with transition metal elements (Cr or Fe), in sharp contrast to conventional 
dilute magnetic semiconductor where free carriers are necessary to mediate the magnetic coupling. 
Magnetic order in two-dimensional thin films gives rise to a topological electronic structure charac- 
terized by a finite Chern number, with quantized Hall conductance e 2 /h. Experimental realization 
of the long sought-after QAH insulator state could enable robust dissipationless charge transport 
at room temperature. 



2 



I. INTRODUCTION 



Soon after the observation of the Hall effect, Hall also observed the anomalous Hall 
effect (AHE)i^ in ferromagnetic metals, where the Hall resistance arises from the spin- 
orbit coupling (SOC) between electric current and magnetic moments, even in the absence 
of external magnetic field. Recent progress on the mechanism of AHE have established a 
link between the AHE and the topological nature of Hall current due to SOC by adopting 
the Berry-phase concepts^"-, in close analogy to the intrinsic spin Hall effect^- 9 . Given the 
experimental discovery of the quantum Hall 7 and quantum spin Hall (QSH) effect o ^ ^ , it is 
natural to ask whether the AHE can also be quantized in a topological magnetic insulator, 
without the external magnetic field and the associated Landau levels. 

A simple mechanism for the QAH insulator has been proposed in a two-band model of a 
two dimensional (2D) magnetic insulator—. In the limit of vanishing spin-orbit coupling, and 
large enough exchange splitting, the majority spin band is completely filled and minority 
spin band is empty. When the exchange splitting is reduced, the two bands intersect each 
other, leading to a band inversion, say near the T point. The degeneracy at the interaction 
region can be removed by turning on the SOC, giving rise to an insulator state with a 
topologically non-trivial band structure characterized by a finite Chern number and chiral 
edge states characteristic of the QAH state (see figures (1) and (2) of Ref.— ). In alternative 
mechanisms of realizing the quantum Hall state without an external magnetic field, bond 
currents on a honeycomb lattice has been proposed^, and the localization of the band 
electron could also give rise to a quantized plateau^. However, these mechanisms seem 
harder to realize experimentally. 

Therefore, the crucial ingredients for realizing an QAH state are: (1) a ferromagneti- 
cally ordered 2D insulator which breaks the time-reversal symmetry; (2) a band inversion 
transition with strong SOC. In comparison with the QSH state, which satisfies the second 
condition^ 1 ^, the first condition of the ferromagnetic order in an insulating state is harder to 
realize. However, the QSH insulators are usually good starting candidates for finding QAH 
effect due to its proximity to the band inversion. Given the QSH effect in HgTe quantum 
wells^'ii through a band inversion transition by varying the thickness of the quantum well, 
the magnetically doped HgMnTe has been proposed as a candidate for the QAH insulator—. 
Unfortunately, the Mn moments do not order spontaneously in HgMnTe, and an additional, 
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small Zeeman field is required for the magnetic alignment of the Mn moments. 

Recently, tetradymite semiconductors Bi 2 Te3, Bi 2 Se3, and Sb2Te3 have been theoretically 
predicted and experimentally observed to be topological insulators with the bulk band gap 
as large as 0.3eV in Bi 2 Se3^r— . The electronic states close to the Fermi level are the bonding 
and anti-bonding p orbitals, and these states undergo a band inversion at the T point when 
the strength of the spin-orbit coupling is increased^. When the thickness is reduced, the 
three dimensional (3D) topological insulator crosses over to a 2D topological insulator in an 
oscillatory fashion, as a function of the thickness^. In this work, we predict that this family 
of compounds doped with proper transition metal elements (Cr or Fe) should give the QAH 
state, the long sought-after quantum Hall state without any external magnetic field. In 
sharp contrast to the ferromagnetic order in conventional dilute-magnetic-semiconductors 
(DMS), where free carriers are necessary to mediate the ferromagnetic couplings via the 
RKKY mechanism22,"— , the magnetic dopants in tetradymite compounds naturally order 
ferromagnetically, mediated by the large spin susceptibility (of the van Vleck type) of the 
host insulator state. Due to the proximity to the band inversion transition, the exchange 
splitting leads to a topologically non-trivial insulating band structure with a finite Chern 
number. The recent experimental progresses on this family of compounds have demon- 
strated that well-controlled layer-by-layer MBE thin film growth can be achieved^— , and 
various transition metal elements (such as Ti, V, Cr, Fe) can be substituted into the parent 
compounds with observable ferromagnetism even above 100K— ~— . By quantitative first- 
principles calculations, we further show that such ferromagnetic QAH insulating state can 
be also reached with Tc as high as around 70K. We will start from simple physical pictures 
based on model analysis, and then conclude by quantitative first-principles calculations. 

II. THE VAN- VLECK PARAMAGNETISM IN BI 2 SE 3 FAMILY COMPOUNDS 

We first discuss minimal requirements for the ferromagnetic insulator phase in a semi- 
conductor system doped with dilute magnetic ions under the assumption that the magnetic 
exchange among local moments is mediated by the band electrons. The whole system can 
then be divided into two sub-systems describing the local moments and band electrons re- 
spectively, which are coupled by a magnetic exchange term. Therefore if we only consider 
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the spatially homogeneous phase, the total free energy of the system can thus be written as, 

Ftotai = \xl}M\ + ]pCe l Mj - J ex M L M e - (M L + M e ) H 

where XL/e is the spin susceptibility of the local moments/electrons, Mlu denotes the mag- 
netization for the local moment and electron sub-system and J ex is the magnetic exchange 
coupling between the local moments and electrons. The onset of the ferromagnetic phase can 
be determined when the minimization procedure of the above free energy gives a non-zero 
magnetization without the external magnetic field H, which leads to J^ x — x^Xe 1 > 0, or 
equivalently xl > -it — • In the dilute limit, we can neglect the direct coupling among the 
local moments, and then the spin susceptibility of the local moment sub-system takes the 
Curie- Weiss form, xl — xfi 2 j/(3kBT), where x is the concentration of the magnetic ions and 
fij is the magnetic moment of a single magnetic ion. Therefore, in order to have non-zero 
FM transition temperature, the above criterion requires a sizable spin susceptibility of the 
electron sub-system % e . In most of the DMS systems, e.g. (Gai_ x Mn x )As, the electronic 
spin susceptibility is negligible for the insulator phase and finite carrier concentration is re- 
quired to conduct the magnetic coupling among local moments. As we shall discuss below, 
unlike the situation in GaAs, sizable spin susceptibility even exists through the Van Vleck 
paramagnetism for the insulator Bi 2 Se3, providing a new mechanism for ordering the doped 
magnetic ions. 

The non-zero paramagnetic spin susceptibility for a band insulator has been known as the 
Van- Vleck paramagnetism^ for a long time. For temperature much less than the band gap, 
the spin susceptibility for a band insulator can be obtained by the second order perturbation 
on the ground state, which can be written as, 

x"= E w, ^MM , (i) 

E nk < K E mk >n ^rnk &nk 

where /j,q is the vacuum permeability and /i# is the Bohr magneton. It is then quite obvious 
that the van- Vleck paramagnetism is caused by the mixing of the conduction and valence 
bands induced by the spin operator in the presence of SOC. Such a mechanism is absent 
in the GaAs system, because the gap in GaAs is between mostly s and p bands and the 
matrix element of S z is nearly zero between conduction and valence bands. In the case of 
Bi 2 Se 3 family, the situation is completely different: both the conduction and valence bands 
are formed by the bonding and anti-bonding p orbitals and the energy gap is opened by the 
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spin-orbital coupling with band inversion, therefore the matrix element of the spin operator 
can be very large. 

The Van-Vleck paramagnetism in Bi 2 Se3 family can be well understood using the follow- 
ing effective k ■ p Hamiltonian derived in ref.—, 
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with k ± = k x ±tk y , e (k) = C+D xy (k 2 x + k 2 y ) +D z kl M(k) = M + B xy (k 2 x + fcj) +B g k^, 
in the basis of |P+,t), \P^,i), \P^,l), |-P^ , T) - The ± in the basis denote the even and 
odd parity states and -If indicate the spin. As discussed before in Ref. — , the four low 
energy states originate from the bonding and anti-bonding combinations of Bi and Se P z 
orbitals, and the parity is a good quantum number at the T point due to the spatial inversion 
symmetry. The topological properties of the Bi2Se3 family can be well explained by the band 
inversion between two sets of Kramer's degenerate states with opposite parity at the T point. 
Here we will show that the band inversion not only gives the non-trivial topological index 
but also increases the spin susceptibility dramatically. For simplicity, we focus on a 2D sheet 
in the k-space with k z = (the situation of general k z is qualitatively similar), where the 
above effective Hamiltonian reduces to two diagonal blocks as^>^ 



H 2D (k) 



(3) 



h{k) 
h* (-k) 

with h {k) = M.(\<i)a z + A xy [k x o x + k y a y ) + e (k). The two eigen-states of h (k) give the 
occupied and unoccupied bands respectively, which can be denoted as |c/c)and \vk). It is 
then obvious that the amplitude of the matrix element (ck\S z \vk) reaches the maximum 
value when Ai(k) = 0. The above condition can only be satisfied when M B xy < 0, which 
is exactly the condition for the band inversion. In the normal phase without inverted bands, 
the maximum value of (ck\ S z \vk) can not be reached and the spin susceptibility is less 
pronounced compared to the system with band inversion. 

To further demonstrate this argument, we perform first-principles calculations (see ref.— 
for method) for the electronic structures and the spin susceptibility of Bi2Se3 bulk. The 
results as function of SOC strength are plotted in Fig. 1(a). When the relative SOC strength 



A/Ao is larger than 0.5, the bands at the T point are inverted, and the spin susceptibility 
starts to increase appreciably around this point. 

III. MEAN FIELD THEORY FOR THE MAGNETIC TOPOLOGICAL INSULA- 
TORS 

In the following, we apply a tight binding version of the Zener model^Sr— to calculate the 
possible Curie temperature at the mean field level. Let's first assume that some magnetic 
local moments are introduced iso-electronically into the Bi 2 Se3 compounds without changing 
the insulating behavior. We show that ferromagnetic ordering with T c as high as 70K can be 
achieved via the van-Vleck paramagnetism, without requiring the existence of free carriers 
as in conventional DMS systems. The total Hamiltonian of our system can be written as 

Hiatal =H LDA + J2 Jeff^l ■ ^u C \,f, C I,u + h.C. (4) 
I /iu,a=xyz 

with 

H LDA = C4m C .7> 
ij,fii/ 

is the single particle Hamiltonian (obtained from first-principles calculations) expressed in 
the basis of local Wannier functions 33 . Here i and j denote the sites, / only runs over all 
the unit cells containing magnetic ions and \w are the orbital/spin indices. The coupling 
strength between the local spin S/ and the band spin s MV of band electrons is described by 
the effective exchange parameter J e ff- 

Performing the mean field approximation and applying the virtual crystal approximation 
(CPA) to average out the spatial inhomogeneity, Eq.pj can be written as 

H MF = H e + H L - xJ eff M a L ■ M a e (5) 

where M£ = (S" =0 ), M" = (s^cI ^Cq^ are the mean field magnetization for the local 
moments and band electrons respectively, with 

a=xyz 

H e = H LDA + ]T J eff M. a L - s%c{^ v + h.c. 

i/iu 

and 

a=xyz 
I 
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We choose the concentration of the magnetic ion to be 5%, the total angular momentum of 
a single magnetic ion to be J = 3/2 and the effective exchange coupling J e ff = 2.0eV, which 
are in the same order with the quantitative first-principles calculations (as described later). 
The above mean field equations have been solved self-consistently, and the ordered moment 
as well as Curie temperature can be calculated. In Figfjjb), the calculated magnetization 
versus temperature are plotted for both bulk material and thin film system with 3, 4, 5 
quintuple layers (QL) thicknesses. The Curie temperature for the bulk system is around 
70K, which can be well compared with previous experimental studies^ - —. The ferromagnetic 
ordering is anisotropic, and the Curie transition is of the Ising type, with a finite transition 
temperature in 2D. Our mean field calculation also indicates that for the thin film system, 
the Curie temperature is only slightly reduced due to the finite size effect, which give rise 
to the possible QAHE as discussed below. 

IV. ELECTRONIC STRUCTURES OF BI 2 SE 3 DOPED WITH TRANSITION 
METAL ELEMENTS 

In this section, we shall show, by parameter-free first principles calculations, that the 
insulating magnetic ground state discussed above can be indeed obtained by proper choice 
of magnetic dopants. It has been suggested by experiments^ - — that the magnetic dopants, 
such as Ti, V, Cr and Fe, will mostly substitute the Bi ions (or the Sb ions in the case of 
Sb2Tes), we therefore concentrate on this situation in the following discussions. Since the 
nominal valence of Bi (or Sb) ions are 3+, a general rule is to find a transition metal element 
which may have a stable 3+ chemical state, so that no free carriers are introduced by this 
iso-electronic substitution. In addition, due to the coexistence of orbital and spin degrees of 
freedom of magnetic dopants (due to the partially filled d-shells), we need a mechanism to 
quench these degrees of freedom and stabilize the insulating state. We shall show that such 
conditions can be satisfied by the combination of crystal field splitting and large Hund's rule 
coupling for Cr and Fe dopants. 

We performed self-consistent first-principle calculations^ for Bi2Se3 doped with various 
transition metal elements, Ti, V, Cr and Fe. We use a large supercell (of size 3x3x1) 
with one of the Bi atoms substituted by a magnetic dopant. The crystal structure and the 
magnetic state are fully optimized to obtain the ground state. The results shown in Fig. 2 
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suggest that a insulating magnetic state is obtained for Cr or Fe doping, while the states are 
metallic for Ti or V doping cases. To understand the results, we first point out that, for all 
the cases, the dopants are nearly in the 3+ valence state, and we always obtain the high-spin 
state due to the large Hund's rule coupling of 3d transition metal ions. Thus the charge and 
spin degrees of freedom are nearly frozen. This will directly lead to the insulating state of 
Fe-doped samples, because the Fe 3+ has five 3d electrons, favoring the d^d 0! ^ configuration 
in a high-spin state, resulting in a gap between the majority and minority spins. We now 
consider the orbitals for other dopants. The local environment of dopants, which substitute 
the Bi sites, is a octahedral formed by six nearest neighboring Se 2 ~ ions. Such a local crystal 
field splits the d-shell into t2 g and e g manifolds. This splitting is large enough to stabilize the 
tHef t° 2g -ef configuration of Cr 3 + ion, resulting in a gap between the t2 9 and e g manifolds. 
For the case of Ti or V doping, even the t<i g manifold is partially occupied, leading to the 
metallic state. It is important to note that, although the local density approximation in the 
density functional theory may underestimate the electron correlation effects, the inclusion 
of electron-electron interaction U (such as in LDA+[7 method) should further enhanced the 
gap. 

The energy gain due to the spin polarization is about 0.9eV/Fe, 1.5eV/Cr, 0.7eV/V and 
0.02eV/Ti, respectively, which are very large numbers except for the case of Ti substitution. 
From the spin splitting of p orbitals of the band electrons, we can estimate the effective ex- 
change coupling J e ff between the local moments and the p electrons^. The estimated value 
is around 2.7eV for Cr and 2.8eV for Fe in Bi2Se3. This exchange coupling is comparable to 
that in GaMnAs^i 

V. THE QUANTIZED ANOMALOUS HALL EFFECT 

Once the ferromagnetically ordered insulating state is achieved with a reasonable T c , 
here in this section, we show that QAH effect can be realized for the 2D thin films. In 
the thick slab geometry, the spatially-separated two pairs of surface states are well defined 
for the top and bottom surfaces respectively. However, with the reduction of the film 
thickness, quantum tunneling between the top and bottom surfaces becomes more and more 
pronounced, giving rise to finite mass term in the effective 2D model^. In the basis of 
|+ t) j| — D )| + 4) )| — t) ) with ± referring to the bonding and anti-bonding combinations 
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H, 



exchange 



(6) 



of the top and bottom surface states, and J/f indicating the spin, the low energy effective 
model for the thin film can be written in the form of H2D as in Eq. [3], with h(k) = 
^■k^z + vp (k x a y — k y a x ), Afc = A + Bk 2 , similar to the Bernevig-Hughes-Zhang model 
describing the low energy physics in HgTe/CdTe quantum well^. When A -B < 0, band 
inversion occurs, and the system will be in the QSH phase if this is the only band inversion 
between two subbands with opposite parity, which is exactly the situation in HgTe/CdTe 
quantum well system. The situation for Bi 2 Se3 film is somewhat different in the sense 
that more pairs of subbands can get inverted at the T point, and the system may still 
remain in the topologically trivial state if the band inversion occurs even number of times^. 
Nevertheless, as we shall show below, regardless of whether the 4-bands system is originally 
in the topologically non-trivial phase or not, a strong enough exchange field will induce QAH 
effect in this system. 

The exchange coupling term induced by the finite magnetization in the FM phase can be 
written as, 

g eff Ma z 
-g e ffMcr : 

with M denoting the strength of the exchange field and g e ff being the effective g-factor of 
the surface states. Added to the 2D model ([3]), the exchange field increases the mass term 
of the upper block and reduces it for the lower block, breaking the time reversal symmetry. 
More importantly, a sufficiently large exchange field can change the Chern number of one 
of the two blocks. As schematically illustrated in Fig. 3, if the 4-band system is originally 
in the topologically trivial phase, the exchange field will induce a band inversion in the 
upper block and push the two sub-bands in the lower block even farther away from each 
other. Therefore the 2D model with a negative mass in the upper block contributes e 2 /h 
for the Hall conductance. On the other hand, if the system is originally in the topologically 
non-trivial phase, both blocks have inverted band structures. In this sufficiently 
large exchange field can increase the band inversion in the upper block and revert the band 
inversion in the lower block. Again the negative mass in the upper block contributes e 2 /h 
for the Hall conductance. These two scenarios are illustrated in Figj3l 

To be realistic, we carried out quantitative calculations for Bi2Se3 films, based on the 
density functional theory and linear response formalism 33 . A spacial uniform exchange field 
is included to take into account the effect of magnetization in the FM state at the mean 
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field level. In FiglJJa), (b) and (c), we plot the lowest four sub-band levels at T point as 
a function of exchange field. The level crossings (indicated by arrows) between the lowest 
conduction bands (blue lines) and valence bands (red lines) are found in all the three Bi 2 Se3 
thin film systems with different layer thickness. The corresponding Hall conductance can be 
obtained using the following Kubo formula 3 , 

Im \(nk\ v x \mk) (mk\ v v Ink)] . s .„ 

°*v = e 2 h ' ' _ /V A /l (n f (E nk ) - n f (E mk )) (7) 

nmk \^nk &mk) 

with E n k being the dispersion of the quantum well sub-bands and v x / y being the velocity 
operators. In the insulator case, where the chemical potential is located inside the energy 
gap between conduction and valence sub-bands, the Hall conductance is determined by the 
first Chern number of the occupied bands and must be an exact integer in the unit of 
e 2 /h. The calculated Hall conductance for Bi 2 Se3 thin film with three typical thickness are 
plotted in figure IU(d) as function of exchange field, where a jump from "0" to "1" in the Hall 
conductance are observed at the corresponding critical exchange field for the level crossing. 

In Fig]5l we plot the sub-band dispersion and corresponding Hall conductance as function 
of chemical potential for 3 and 5 QL Bi2Se3 thin films with different exchange fields. We 
can find that the change induced by the exchange field is mainly in the area near T point. 
For the QAH phases ((d) and (h)), non-zero integer plateau is observed when the chemical 
potential falls into the energy gap. The difference between the 3 and 5 QL cases suggests 
that the QAH plateau is much narrower for thick film due to the narrower energy gap coming 
from the intersection of top and bottom surface states. 

In conclusion we have shown that magnetic dopants Cr and Fe form long range magnetic 
order in the insulating state of the Bi2Se3 family of topological insulators. The ferromagnetic 
order lead to a topologically non-trivial electronic structure with quantized Hall conductance 
without any external magnetic field. In real samples, a small amount of bulk carriers could 
always be present, however, if the concentration is low enough, they will be localized by 
disorder in two dimensions, and will not affect the precise quantization of the Hall plateau. 
The edge states of our QAH state conducts charge current without any dissipation in the 
absence of any external magnetic field. This effect could enable a new generation of low 
power electronic devices. 
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FIG. 1: The calculated spin susceptibility and Curie temperature for Bi2Se3. (a) The 

Van-Vleck type spin susceptibility of Bi2Se3 bulk as a function of spin-orbital-coupling (SOC) 
strength. Ao is the real SOC strength, and it is uniformly scaled to simulate various SOC strength 
A. When A/Ao is larger than 0.5, the energy bands at T are inverted, and Bi2Se3 becomes strong 
topological insulator, (b) The spontaneous magnetization M as a function of temperature for 
Cr-doped (S z =3/2) Bi2Se3 bulk as well as thin films with thickness of 3, 4 and 5 quintuple layers 
(QL). 
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FIG. 2: The calculated density of states (DOS) for Bi2Se3 doped with different transi- 
tion metal elements. The Fermi level is located at energy zero, and the positive and negative 
values of DOS are used for up and down spin, respectively. The blue lines are projected partial 
DOS of the 3d states of transition metal ions. It is shown that Cr or Fe doping will give the 
insulating magnetic state. 
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FIG. 3: Evolution of the sub-bands structure upon increasing the exchange field. The 

solid (dashed) lines denote the sub-band which have even (odd) parity at T point, and the blue 
(red) color denotes the spin down (up) electrons, (a) The initial sub-bands are not inverted. When 
a exchange field is introduced, the spin down sub-bands (blue lines) are pushed down and the spin 
up sub-bands (red lines) are lifted up. When the exchange field is strong enough, a pair of inverted 
sub-bands appear (the red dash line and the blue solid line), (b) The initial sub-bands are already 
inverted at T point. The exchange field reverts the band inversion in one pair of sub-bands (the red 
solid line and the blue dashed line) and increase the band inversion in the other pair of sub-bands 
(the red dash line and the blue solid line). The SOC opens a gap at the cross points of spin-up and 
spin-down subbands as shown on the right most panel of (a) and (b), leading to the QAH state by 
the mechanism proposed in Ref t 12 ' 15 
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FIG. 4: The quantized anomalous Hall(QAH) conductance, (a), (b) and (c) are the lowest 
sub-bands at the T point plotted versus the exchange field, for Bi2Se3 films with thickness of 3, 
4 and 5QL, respectively. The red (blue) lines denote the occupied (unoccupied) states. With 
the increasing of the exchange field, a switch of occupied and unoccupied states happens (i.e, a 
quantum phase transition to the QAH phase) at the critical field strength indicated by arrows, (d), 
The calculated Hall conductance using the Hamiltonian from first-principles calculations under an 
uniform exchange field (see text for the detail) for 3, 4 and 5QL Bi2Se3 films. As we expect, the 
Hall conductance is zero before the QAH transition, but a xy = e 2 /h afterwards. 
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FIG. 5: The sub-bands dispersion and the Hall conductance. The sub-bands dispersion 
of 2D Bi2Se3 thin film are plotted for (a) 3QL slab at exchange field=-0.05eV, (c) 3QL slab at 
exchange field=-0.24eV, (e) 5QL slab at exchange field=-0.01eV, and (g) 5QL slab at exchange 
field=-0.04eV. The insets of (e) and (g) zoom out the dispersion near T point. The corresponding 
Hall conductance are plotted in the right panels ((b), (d), (f) and (g)) as the function of the 
chemical potential. The systems in (a) and (e) are normal insulators, which have zero platforms 
in Hall conductance when the chemical potential locates in the energy gap. The system in (c) and 
(g) are QAH insulators, in which the Hall conductance have quantized value: e 2 /h. 
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